Information processing device, information processing method, and computer program

ABSTRACT

An information processing device includes: an image acquirer configured to acquire a shot image of a user from an imaging device; a registered user information holder configured to hold face identification data of a registered user; a face authentication section configured to execute authentication processing of a face image in the shot image by using the face identification data held in the registered user information holder; a shooting condition controller configured to adjust a shooting condition to bring a characteristic of the face image close to a characteristic of a face image of an acquisition source of the face identification data and make the imaging device perform shooting in order to improve an authentication result in the face authentication section; and an information processing section configured to execute information processing on the basis of an authentication result of a face image obtained by adjustment.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present disclosure relates to an information processing device thatexecutes information processing by using a shot image, an informationprocessing method thereof, and a computer program.

2. Description of the Related Art

In recent years, it is becoming general to equip a personal computer, agame machine, etc. with a camera and image the figure of a user to usethe taken image in various forms. For example, systems in which an imageof a user is transmitted to the other side as it is via a network, suchas television telephone and video chat, and systems in which the motionof a user is recognized by image analysis and the recognized motion isused as input information of a game or information processing have beenput into practical use (e.g. refer to WO 2007/050885 A2). Moreover, inrecent years, it is becoming possible to realize games and imageexpression giving a user a more feeling of being present in the realworld by detecting the motion of an object in a three-dimensional spaceincluding the depth direction with high accuracy.

In the case of shooting a space where a wide variety of objects existand executing information processing with use of the shot image as inputdata, the accuracy of the information processing is more susceptible tothe shooting environment and so forth compared with operation throughbuttons of an input device or a graphical user interface (GUI).Therefore, it is desired to realize a device that can execute stableinformation processing even when the environment changes. Particularlywhen an individual user is associated with the figure of the user in ashot image and an authentication is carried out at the time of login orthe like, it is preferable that accurate associating is made with asmall burden of the user.

RELATED ART LIST

WO 2007/050885 A2

SUMMARY OF THE INVENTION

There is a need for the present disclosure to provide a technique thatallows execution of authentication processing with use of a shot imagewith a small burden of the user and stable accuracy.

According to an embodiment of the present disclosure, there is providedan information processing device. The information processing deviceincludes an image acquirer configured to acquire a shot image of a userfrom an imaging device, a registered user information holder configuredto hold face identification data of a registered user, and a faceauthentication section configured to execute authentication processingof a face image in the shot image by using the face identification dataheld in the registered user information holder. The informationprocessing device further includes a shooting condition controllerconfigured to adjust a shooting condition to bring a characteristic ofthe face image close to a characteristic of a face image of anacquisition source of the face identification data and make the imagingdevice perform shooting in order to improve an authentication result inthe face authentication section, and an information processing sectionconfigured to execute information processing on a basis of anauthentication result of a face image obtained by adjustment.

According to another embodiment of the present disclosure, there isprovided an information processing method. The information processingmethod includes, by an information processing device, acquiring a shotimage of a user from an imaging device, reading out face identificationdata of a registered user stored in a storage device and carrying outauthentication of a face image in the shot image by using the faceidentification data, adjusting a shooting condition to bring acharacteristic of the face image close to a characteristic of a faceimage of an acquisition source of the face identification data andmaking the imaging device perform shooting in order to improve anauthentication result in the carrying out authentication, and executinginformation processing on a basis of an authentication result of a faceimage obtained by adjustment.

According to another embodiment of the present disclosure, there isprovided a computer program for a computer. The computer programincludes acquiring a shot image of a user from an imaging device,reading out face identification data of a registered user stored in astorage device and carrying out authentication of a face image in theshot image by using the face identification data, adjusting a shootingcondition to bring a characteristic of the face image close to acharacteristic of a face image of an acquisition source of the faceidentification data and making the imaging device perform shooting inorder to improve an authentication result in the carrying outauthentication, and executing information processing on a basis of anauthentication result of a face image obtained by adjustment.

What are obtained by translating arbitrary combinations of the aboveconstituent elements and expressions of the present disclosure amongmethod, device, system, recording medium, computer program, and so forthare also effective as embodiment of the present disclosure.

According to the embodiment of the present disclosure, the accuracy ofauthentication processing with use of a shot image can be easilystabilized.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram showing an information processing system in anembodiment of the present disclosure;

FIG. 2 is a diagram showing the internal configuration of an informationprocessing device in the embodiment;

FIG. 3 is a diagram showing the functional block configuration of theinformation processing device in the embodiment;

FIG. 4 is a diagram showing one example of a space shot by a camera inthe embodiment;

FIG. 5 is a diagram showing the result of face identification by a faceauthentication section in the embodiment;

FIG. 6 is a diagram showing a login screen including face framesdisplayed on an output device in the embodiment;

FIG. 7 is a diagram showing the login screen in which a user puts theface in a face frame in the embodiment;

FIG. 8 is a diagram schematically showing data stored in a registereduser information holder and a provisionally-registered data holder inthe embodiment;

FIG. 9 is a flowchart showing the procedure of processing of registeringface identification data by a face identification data registrationsection in the embodiment;

FIG. 10 is a diagram for explaining a setting example of a registrationcondition and a provisional registration condition in the embodiment;

FIG. 11 is a flowchart showing the procedure of processing of formallyregistering provisionally-registered data by a registration determinerin the embodiment;

FIG. 12 is a diagram for explaining a setting example of a reliabilityupdate condition in the embodiment;

FIG. 13 is a diagram schematically showing a data example of reliabilityin the embodiment;

FIG. 14 is a diagram for explaining the basic adjustment operation of ashooting condition controller in the embodiment; and

FIG. 15 is a flowchart showing the procedure of processing of carryingout face authentication by the face authentication section whileadjusting the shooting condition of the camera by the shooting conditioncontroller in the embodiment.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

FIG. 1 shows an information processing system 1 according to anembodiment of the present disclosure. The information processing system1 includes an information processing device 10 as a user terminal and aserver 5. An access point (hereinafter, referred to as “AP”) 8 hasfunctions of a wireless access point and a router. The informationprocessing device 10 connects to the AP 8 in a wireless or wired mannerand communicably connects to the server 5 on a network 3.

An auxiliary storage device 2 is a high-capacity storage device such asa hard disc drive (HDD) or a flash memory. It may be an external storagedevice that connects to the information processing device 10 by auniversal serial bus (USB) or the like or may be a built-in storagedevice. An output device 4 may be a television having a display tooutput images and a speaker to output sounds or may be a computerdisplay. The output device 4 may be connected to the informationprocessing device 10 by a wiring cable or may be wirelessly connectedthereto.

The information processing device 10 connects to an input device 6operated by a user in a wireless or wired manner and the input device 6outputs an operation signal indicating an operation result by the userto the information processing device 10. When accepting the operationsignal from the input device 6, the information processing device 10reflects it in processing of an operating system (OS, i.e. systemsoftware) or an application and makes the processing result be outputfrom the output device 4. The input device 6 has plural input parts suchas plural push operation buttons, analog sticks with which an analogamount can be input, and pivotal buttons.

When accepting the operation signal from the input device 6, theinformation processing device 10 reflects it in processing of anapplication and makes the processing result be output from the outputdevice 4. In the information processing system 1, the informationprocessing device 10 is a game device that executes a game and each ofthe input devices 6 a and 6 b (hereinafter, often referred to as theinput device 6 generically) is an apparatus, such as a game controller,to provide the operation signal of a user to the information processingdevice 10. The input device 6 may be an input interface such as akeyboard and a mouse. A camera 7 as an imaging device is provided nearthe output device 4 and images a space around the output device 4.Although an example in which the camera 7 is attached to an upper partof the output device 4 is shown in FIG. 1, it may be disposed at a sidepart of the output device 4. In any case, the camera 7 is disposed at aposition at which it can image a user located in front of the outputdevice 4. The camera 7 may be a stereo camera.

The server 5 provides network services to users of the informationprocessing system 1. The server 5 manages network accounts to identifythe respective users and each user signs in the network servicesprovided by the server 5 by using the network account. By signing in thenetwork services from the information processing device 10, the user canregister, in the server 5, save data of a game and trophies as virtualprizes won in game play.

In FIG. 1, a situation in which two users A and B are operating theinput devices 6 a and 6 b as game controllers is shown. The users A andB operate the input devices 6 a and 6 b, respectively, to input apasscode for login. After being authenticated by the informationprocessing device 10, they log in to the OS of the informationprocessing device 10 and thereby can enjoy an application such as agame.

In recent years, a game in which the motion of a user is reflected inthe motion of a game character has also appeared. In a game utilizinggesture of a user, the user does not need to hold the input device 6 andcan intuitively move a character. In such a game, because the user doesnot use the input device 6 in the first place, it is preferable thatuser authentication is executed without use of the input device 6 alsowhen the user logs in to the OS of the information processing device 10.It is meaningful in the information processing system 1 that the usercan log in through simple user authentication irrespective of the kindof game to be played by the user after the login. Therefore, in theinformation processing system 1 of the present embodiment, a techniqueby which user authentication can be easily executed by using a takenimage by the camera 7 is provided.

FIG. 2 shows the internal configuration of the information processingdevice 10. The information processing device 10 has a main power button20, a power-ON light emitting diode (LED) 21, a standby LED 22, a systemcontroller 24, a clock 26, a device controller 30, a media drive 32, aUSB module 34, a flash memory 36, a wireless communication module 38, awired communication module 40, a sub-system 50, and a main system 60.

The main system 60 includes a main central processing unit (CPU), amemory as a main storage device, a memory controller, a graphicsprocessing unit (GPU), and so forth. The GPU is used mainly forarithmetic processing of a game program. These functions may beconfigured as a system on a chip and formed on one chip. The main CPUhas functions to activate an OS and execute an application installed inthe auxiliary storage device 2 under an environment provided by the OS.

The sub-system 50 includes a sub-CPU, a memory as a main storage device,a memory controller, and so forth and does not include a GPU. The numberof circuit gates of the sub-CPU is smaller than the number of circuitgates of the main CPU and the operating power consumption of the sub-CPUis lower than that of the main CPU. The sub-CPU operates in a periodduring which the main CPU is in the standby state and its processingfunctions are limited in order to suppress the power consumption. Thesub-CPU and the memory may be formed on different chips.

The main power button 20 is an input part to which an operation inputfrom a user is made. It is provided on the front surface of a casing ofthe information processing device 10 and is operated to turn on or offpower supply to the main system 60 of the information processing device10. Hereinafter, that the main power supply is in an on-state means thatthe main system 60 is in the active state, and that the main powersupply is in an off-state means that the main system 60 is in thestandby state. The power-ON LED 21 is lit when the main power button 20is switched on and the standby LED 22 is lit when the main power button20 is switched off.

The system controller 24 detects pressing-down of the main power button20 by the user. If the main power button 20 is pressed down when themain power supply is in the off-state, the system controller 24 acquiresthe pressing-down operation as an “on-instruction.” On the other hand,if the main power button 20 is pressed down when the main power supplyis in the on-state, the system controller 24 acquires the pressing-downoperation as an “off-instruction.”

The main CPU has a function to execute game programs installed in theauxiliary storage device 2 and a read-only memory (ROM) medium 44whereas the sub-CPU does not have such a function. However, the sub-CPUhas a function to access the auxiliary storage device 2 and a functionto transmit and receive data to and from the server 5. The sub-CPU hasonly such limited processing functions and therefore can operate atrelatively low power consumption compared with the main-CPU. Thesefunctions of the sub-CPU are executed when the main-CPU is in thestandby state. Because the sub-system 50 is operating when the mainsystem 60 is in the standby state, the information processing device 10of the present embodiment keeps being in the sign-in state in thenetwork services provided by the server 5.

The clock 26 is a real-time clock. It generates present date-and-timeinformation and supplies it to the system controller 24, the sub-system50, and the main system 60.

The device controller 30 is a large-scale integrated circuit (LSI) thatcarries out exchange of information among devices like a southbridge. Asshown in the diagram, to the device controller 30, devices areconnected, such as the system controller 24, the media drive 32, the USBmodule 34, the flash memory 36, the wireless communication module 38,the wired communication module 40, the sub-system 50, and the mainsystem 60. The device controller 30 absorbs the differences in theelectrical characteristics and the data transfer rate among therespective devices and controls the timing of data transfer.

The media drive 32 is a drive device that drives the ROM medium 44 thatis loaded therein and in which application software such as a game andlicense information are recorded and reads out a program, data, and soforth from the ROM medium 44. The ROM medium 44 is a read-only recordingmedium such as an optical disc, a magneto-optical disc, or a Blu-ray(registered trademark) disc.

The USB module 34 is a module that connects to an external apparatus bya USB cable. The USB module 34 may connect to the auxiliary storagedevice 2 and the camera 7 by USB cables. The flash memory 36 is anauxiliary storage device forming an internal storage. The wirelesscommunication module 38 wirelessly communicates with e.g. the inputdevice 6 based on a communication protocol such as the Bluetooth(registered trademark) protocol or the IEEE 802.11 protocol.

The wireless communication module 38 may be compatible with a thirdgeneration digital mobile phone system that complies with theInternational Mobile Telecommunication 2000 (IMT-2000) standard definedby the International Telecommunication Union (ITU) and furthermore maybe compatible with a digital mobile phone system of another generation.The wired communication module 40 communicates with an externalapparatus in a wired manner and connects to the network 3 via the AP 8for example.

In the information processing system 1 of the present embodiment, if theuser presses down the main power button 20 when the informationprocessing device 10 is in the power-off state, the informationprocessing device 10 turns on the main power supply to activate the OS(system software) and execute login processing for the user. In thislogin processing, the information processing device 10 functions as aface authentication system using an image taken by the camera 7.

FIG. 3 shows the functional block configuration of the informationprocessing device 10. Although the information processing device 10 hasa function to execute various kinds of information processing such as agame after the login processing, only functional blocks relating to thelogin processing including authentication are shown in FIG. 3. Theinformation processing device 10 has an input acceptor 102, an imageacquirer 104, a shooting condition controller 106, a login controller110, a registered user information holder 130, and a face identificationdata registration section 150. The login controller 110 has a takenimage display section 112, a face authentication section 114, a faceframe processing section 116, and a login processing section 120. Theface identification data registration section 150 has a data generator152, a registration determiner 154, and a provisionally-registered dataholder 156.

The respective elements described as functional blocks that executevarious kinds of processing in FIG. 3 can be formed with circuit block,memory, and other LSIs in terms of hardware and are implemented by aprogram loaded into the memory and so forth in terms of software.Therefore, it will be understood by those skilled in the art that thesefunctional blocks can be implemented in various forms by only hardwareor only software or a combination of them, and they are not limited toany.

One of characteristics of the information processing device 10 of thepresent embodiment is that it assists simple login operation of theuser. To log in to the OS of the information processing device 10,basically the user acquires a user account in advance and registers itin the information processing device 10. Hereinafter, the user who hasregistered the user account in the information processing device 10 willbe referred to as the “registered user.”

The input acceptor 102 accepts operation information from a user via theinput device 6 and supplies this information to other functional blocksaccording to need. The image acquirer 104 acquires a taken imageobtained by imaging by the camera 7 and stores it in a memory. Thecamera 7 shoots a spatial image at a predetermined cycle, specificallyshoots one spatial image per 1/30 seconds for example, and provides thetaken images to the image acquirer 104. The camera 7 is so disposed thatits optical axis is oriented in the front direction of the output device4, and thereby shoots a user who exists in front of the output device 4.

The login controller 110 extracts a region in which a face is capturedin the shot image acquired by the image acquirer 104 and carries out aface authentication by using feature data of the face represented inthis region. Then, if the authentication succeeds, i.e. if it isdetermined that the shot face is the face of a registered user, thelogin controller 110 allows this user to log in.

The registered user information holder 130 holds various pieces ofinformation relating to the registered user. Specifically, it holds faceidentification data, the online identification (ID) (nickname on thenetwork) of the user, a login passcode, and so forth as registered userinformation in association with the user account. The faceidentification data may be feature data of a face image of theregistered user or face image data itself.

The face identification data is data employed as a comparison target inface authentication processing in the login controller 110. It isgenerated in accordance with an employed face recognition algorithm andis stored in the registered user information holder 130 by the faceidentification data registration section 150. For example, the faceidentification data may be data obtained by extracting, ascharacteristics, the relative positions and sizes of parts of a face andthe shapes of eyes, nose, cheekbone, and jaw. Furthermore, the faceidentification data may be data representing the difference from anaverage face. In addition, it may be a statistic representing thedistribution of the luminance vector and so forth. What kind of faceidentification data is to be extracted is determined by the employedface recognition algorithm.

The shooting condition controller 106 controls the shooting condition ofthe camera 7 so that a shot image at the time of login processing mayget close to a shot image when face identification data held by theregistered user information holder 130 is acquired. Specifically, theshooting condition controller 106 adjusts the shooting condition so thatcharacteristics that change through the adjustment of the shootingcondition, such as the luminance distribution and the frequencydistribution, among various kinds of characteristics possessed by theshot image may get close to those of the shot image when the faceidentification data is acquired. This suppresses the deterioration ofthe authentication accuracy attributed to change in the shootingenvironment and so forth.

In the present embodiment, the functions of the taken image displaysection 112, the face frame processing section 116, and the faceidentification data registration section 150 are implemented by a faceauthentication application. The functions of the face authenticationsection 114 are implemented by a face recognition engine. They areautomatically activated by the OS when the main power button 20 ispressed down. The face authentication application and the facerecognition engine may be configured as one application.

First, an example of login processing to which the present embodimentcan be applied will be described. However, the present embodiment can beapplied to any technique of carrying out authentication by comparisonbetween registered image data and a shot image and does not intend tolimit its application range to the login processing to be describedbelow. In this example, at least users A and B exist. The online ID ofthe user A is “HANAKO” and the online ID of the user B is “SACHIKO.”

When the user presses down the main power button 20 of the informationprocessing device 10, the main power supply of the informationprocessing device 10 is turned on and the input acceptor 102 acceptsinformation on the pressing-down of the main power button 20 as a loginrequest from the user. The login request may be accepted via the inputdevice 6. When the input acceptor 102 accepts the login request, therespective functions in the login controller 110 are implemented. Thelogin controller 110 has a function to determine whether or not topermit login of the user on the basis of the result of face recognitionof the user.

When the input acceptor 102 accepts the login request, the taken imagedisplay section 112 reads out a taken image acquired by the imageacquirer 104 from the memory and displays it on the output device 4,which is a display. A live image shot by the camera 7 is therebydisplayed on the output device 4, so that the user present in front ofthe output device 4 is displayed on the output device 4.

FIG. 4 shows one example of the space shot by the camera 7. In this shotspace, three users exist. A rectangular frame surrounding the users inFIG. 4 expresses the imaging range of the camera 7. The imaging range ofthe camera 7 defines the range displayed on the output device 4 as thelive image but the live image may be part of the taken image. The faceauthentication section 114 extracts a part estimated to be a person'sface in the taken image and derives feature data of this part. The faceauthentication section 114 then compares the derived feature data withface identification data held in the registered user information holder130 and determines whether or not the extracted face is the face of aregistered user.

Specifically, the face authentication section 114 derives the degrees ofmatch between the feature data of the extracted face image of the userand the face identification data of all registered users held in theregistered user information holder 130. This degree of match isnumerically expressed. For example, the degree of match is derived inthe form of a score out of 100. If the degree of match of a registeredface image with the feature data surpasses a predeterminedauthentication threshold such as 90, the face authentication section 114determines that the imaged user is a registered user and identifieswhich registered user the imaged user is.

If plural users whose degree of match surpasses the authenticationthreshold exist, the face authentication section 114 may determine thatthe imaged user is the registered user of the face identification datawith which the best score is derived. If the face identification datawhose degree of match surpasses the authentication threshold does notexist as the result of derivation of the degrees of match between thefeature data of the face image of the user extracted from the takenimage and the face identification data of all registered users, the faceauthentication section 114 determines that the user included in thetaken image is not the registered user. By doing this, the faceauthentication section 114 executes the authentication processing byassociating the face image in the taken image with the registered userwith use of the face identification data held in the registered userinformation holder 130. In the present embodiment, this processing ispositioned as a first-stage face authentication.

FIG. 5 shows the result of face identification by the faceauthentication section 114. Here, it is determined that the left user isthe user A and the right user is the user B and the middle user is not aregistered user. The face authentication section 114 sets a face region200 a indicating the position of the face of the user A (online ID:HANAKO) in the taken image and a face region 200 b indicating theposition of the face of the user B (online ID: SACHIKO) in the takenimage, and provides the face frame processing section 116 withinformation to identify the position coordinates of the face regions 200a and 200 b and the imaged registered users.

Hereinafter, an example will be shown in which the position coordinatesare expressed by two-dimensional coordinates when the taken image isdisplayed on the display. However, the position coordinates may becoordinates defined on a video random access memory (VRAM). In any case,it is enough that the coordinates of the face regions 200 a and 200 b(hereinafter, often referred to as the face region 200 generically) andthe coordinates of face frames to be described later are expressed on acommon coordinate system.

Each face region 200 may be set as a rectangular region in contact withthe contour of the face in the taken image. Alternatively, it may be setas a rectangular region slightly wider than the facial contour. Here,the contour of the face means a contour including the head hair.However, the contour of the face may be set excluding the head hair ifthe head hair is not taken into consideration in face recognitionprocessing of the face authentication section 114 for example. The sizeand shape of the face region 200 are determined by the size and shape ofthe face of the user in the taken image. Therefore, the size and shapeof the face region 200 differ for each user. Furthermore, even for thesame user, the size and shape of the face region 200 change depending onthe distance from the camera 7.

The information that is provided from the face authentication section114 to the face frame processing section 116 and is to identify theregistered user may be the user account of the registered user or may bethe online ID. The face authentication section 114 provides the faceframe processing section 116 with the position coordinates of the faceregion 200 in association with the registered user identificationinformation. Specifically, in the example shown in FIG. 5, the faceauthentication section 114 provides the face frame processing section116 with a combination of the face region 200 a and the identificationinformation of the user A and a combination of the face region 200 b andthe identification information of the user B.

The face frame processing section 116 displays a face frame on theoutput device 4 for the registered user detected by the faceauthentication section 114. This face frame is displayed in order forthe registered user to move the face and dispose it in the face framewhen logging in. Therefore, the registered user is allowed to log in tothe information processing device 10 by putting the user's own face inthe face frame displayed on the output device 4.

FIG. 6 shows a login screen including face frames displayed on theoutput device 4. The face frame processing section 116 displays faceframes 210 a and 210 b (hereinafter, often referred to as the face frame210 generically) for registered users on the basis of the informationthat is provided from the face authentication section 114 and is toidentify the position coordinates of the face regions 200 a and 200 band the imaged registered users. In this example, the face frameprocessing section 116 displays the face frame 210 a for the user A anddisplays the face frame 210 b for the user B. At this time, the faceframe processing section 116 displays the online ID of the user A nearthe face frame 210 a and displays the online ID of the user B near theface frame 210 b. This allows the users A and B to come to know that theown face is properly recognized and prompts them to move the face intothe face frames 210 a and 210 b.

If an online ID different from the own online ID is displayed near theface frame 210 displayed near the own face, the user can come to knowthat the face recognition is not being properly carried out andtherefore does not make action of putting the face in the face frame210. Because the middle user is not a registered user, the face frame210 is not displayed. However, in the case of recognizing anunregistered user as a guest user as described later, the face frame 210for this user is also displayed in some cases.

The face frame processing section 116 gives an ID to each of the faceframes 210 and provides the face authentication section 114 with faceframe IDs, the position coordinates of the face frames 210, and theidentification information of the users for which the face frames 210are displayed. The position coordinates of the face frame 210 providedto the face authentication section 114 by the face frame processingsection 116 may be the position coordinates of the face frame 210 itselfor may be the coordinates of a rectangle circumscribed about the faceframe 210. Hereinafter, the position coordinates of the face frame 210itself and the position coordinates of a rectangle circumscribed aboutthe face frame 210 will be referred to as the position coordinates ofthe face frame collectively. The position coordinates of the face frameare used to detect a face image of the user in the face authenticationsection 114.

For example, the face frame processing section 116 sets “ID1” as theface frame ID of the face frame 210 a and sets “ID2” as the face frameID of the face frame 210 b. The face frame processing section 116provides the face authentication section 114 with a combination of“ID1,” the position coordinates of the face frame 210 a, and theidentification information of the user A and a combination of “ID2,” theposition coordinates of the face frame 210 b, and the identificationinformation of the user B. Furthermore, the face frame processingsection 116 provides the login processing section 120 with the faceframe IDs and the identification information of the users for which theface frames 210 are displayed. Therefore, in this case, the face frameprocessing section 116 provides the login processing section 120 with acombination of “ID1” and the identification information of the user Aand a combination of “ID2” and the identification information of theuser B.

FIG. 7 shows a login screen in which a user has put the face in a faceframe. Here, a state is shown in which the user A has moved the face andbody in such a manner that the face enters the face frame 210 adisplayed on the output device 4. The face authentication section 114monitors whether a person's face is put in the face frame 210. If a faceis put therein, the face authentication section 114 determines whetherthe face put in the face frame 210 is the face of the registered user byusing face identification data held in the registered user informationholder 130.

The face authentication section 114 can monitor whether a person's faceis put in the face frame 210 on the basis of the position coordinates ofthe face frame 210 provided from the face frame processing section 116.The face recognition algorithm is as described above. When estimatingthat a person's face is included in the face frame 210, the faceauthentication section 114 derives the feature data of this part andcompares the feature data with face identification data held in theregistered user information holder 130 to determine that the extractedface is the face of the registered user.

The face authentication section 114 has been notified of the combinationof the face frame ID, the position coordinates of the face frame 210,and the identification information of the user for which the face frame210 is displayed from the face frame processing section 116, andcompares the feature data of the face image of the person included inthe face frame 210 with the face identification data of the user forwhich the face frame 210 is displayed. Because having been notified ofthe user that should be included in the face frame 210 in advance, theface authentication section 114 does not need to compare the featuredata of the face included in the face frame 210 with the faceidentification data of all registered users and thus can efficientlyexecute the face recognition processing.

At this time, the face authentication section 114 may determine that theface of the registered user is put in the face frame 210 by detectingthat the face of the registered user has been put in the face frame 210for a predetermined time (e.g. several seconds). As a result, in theexample of FIG. 7, the face authentication section 114 determines thatthe face that has entered the face frame 210 a is the face of the userA. In the present embodiment, this processing is positioned assecond-stage face authentication.

Through the first-stage and second-stage face authentications, the userauthentication at the time of login ends. The action of putting a facein the face frame 210 by a user is made on the basis of the user'sintention to log in. When the user does not desire to log in, the userdoes not need to put the face in the face frame 210. As above, in thepresent embodiment, the registered user who will possibly log in isdetected by the first-stage face authentication and the registered userhaving an intention to log in is detected by the second-stage faceauthentication. The registered user is authenticated by only carryingout simple operation of putting the face in the face frame 210. Thus,the working burden on the user at the time of login can be made verysmall.

When detecting that the face of the user A has entered the face frame210 a, the face authentication section 114 notifies the login processingsection 120 of the face frame ID to identify the face frame 210 a andthe user identification information to identify the user A. As alreadydescribed, the login processing section 120 has been notified of theface frame IDs and the identification information of the users for whichthe face frames 210 are displayed from the face frame processing section116 in advance. When being notified of the face frame ID and the useridentification information from the face authentication section 114, thelogin processing section 120 extracts the user identificationinformation associated with the face frame ID notified from the faceframe processing section 116 and determines the match between the piecesof user identification information. Here, corresponding to the faceframe ID of ID1, the identification information of the user A isnotified as both of the pieces of identification information from theface authentication section 114 and the face frame processing section116. Therefore, the login processing section 120 recognizes that theface of the user A is detected in the face frame 210 a displayed for theuser A. Due to this, the login processing section 120 allows the user Ato log in to the information processing device 10.

In the above-described example, the face authentication section 114compares the face identification data of the one user who should beincluded in the face frame 210 with the feature data of the faceincluded in the face frame 210. However, the face authentication section114 may compare the face identification data of all registered userswith the feature data of the face included in the face frame 210. As aresult, the face authentication section 114 often detects the face of aregistered user other than the user A in the face frame 210 a displayedfor the user A. The face authentication section 114 notifies the loginprocessing section 120 of the user identification information toidentify the user included in the face frame 210 and the face frame ID,and the login processing section 120 compares them with the face frameID and the identification information of the user for which the faceframe 210 is displayed, notified from the face frame processing section116 in advance.

If the identification information of the user for which the face frame210 is displayed is different from the identification information of theuser actually detected in the face frame 210 regarding the same faceframe ID, i.e. if the face of a registered user other than the user A isdetected in the face frame 210 a displayed for the user A, the loginprocessing section 120 determines that the face image detected in theface frame 210 a is not the face image of the user A, and prevents theuser A or the registered user who puts the face in the face frame 210 afrom logging in.

As described above, in the present embodiment, the login controller 110causes a registered user to log in after carrying out the userauthentication by face authentications of two stages. The first-stageface authentication and the second-stage face authentication may beconcurrently executed at different cycles. For example, the first-stageface authentication and the second-stage face authentication areconcurrently operated at a cycle of one second and a cycle of 1/60seconds, respectively. Due to this, particularly when a large number ofusers exist at a time, the speed of the login processing can be enhancedcompared to the case in which the face authentications of the two stagesare executed one person by one person. By setting the operating cycle ofthe first-stage face authentication long and setting the operating cycleof the second-stage face authentication short as described above,transition from the first stage to the second stage can be rapidly madewith a suppressed processing burden.

After login of one registered user through the second-stage faceauthentication processing, if a registered user who has not yet loggedin is being shot, the login screen may continue to be displayed untilthis registered user logs in. In this case, it is preferable for theface frame processing section 116 to erase the face frame 210 displayedfor the user who has logged in from the output device 4. If there is alimit that only one registered user is allowed to log in through thisface authentication, transition to the home screen provided by the OS ofthe information processing device 10 or the like may be made after loginof one registered user.

If the face authentication section 114 cannot detect a face in the faceframe 210 for a predetermined time after the face frame 210 is displayedin the login screen, the login processing by face authentication may endand transition to login processing by use of the input device 6 may bemade. Furthermore, if the user does not desire the login processing byface authentication, the login processing by face authentication may beended by using the input device 6 for example and transition to loginprocessing by use of the input device 6 may be made.

The face identification data used here differs depending on the facerecognition algorithm employed by the face authentication section 114 asdescribed above and may be either feature data or face image dataitself. As one of face recognition algorithms, there is a method inwhich the degree of match is calculated on the basis of a differenceimage between a shot face image and a face image registered in advanceand the distance from an eigenspace of a difference image group acquiredin advance (refer to e.g. Japanese Patent Laid-Open No. 2002-157596).

Not only in this technique but in authentication based on a pixel valueset, such as authentication in which block matching is performed betweenan image shot at the time of the authentication and a registered imageand authentication in which probability density is obtained on the basisof the luminance distribution vector of an image with a statistic suchas a covariance matrix, the face image as the source of the registereddata readily affects the authentication accuracy. For example, possiblythe overall luminance and the angle of light shining on a face greatlydiffer between an image shot under natural light incident from a windowand an image shot under indoor illumination at night.

When these images are compared, the accuracy of derivation of the degreeof match and hence the authentication accuracy tend to be low comparedwith when images shot under the same illumination environment arecompared. Such a change in the illumination environment is caused byvarious factors such as time zone, weather, whether a curtain is openedor closed, and which illumination lamp is lit. The authenticationaccuracy possibly changes due to change in not only the illuminationenvironment but also the user oneself, such as the degree of facialswelling, aging change, whether make-up is done, and hairstyle.

If the authentication accuracy changes due to such external factors anda user has difficulty in being recognized or is recognized as anotherperson even though being a registered user, the user is given a lot ofstress. Not only that, possibly an inconvenience that informationprocessing to be executed subsequently, such as a game, is not properlyexecuted and so forth occur. Therefore, the information processingdevice 10 of the present embodiment suppresses the occurrence of theabove-described problem by executing at least one of the following kindsof processing.

(1) automatic registration of face identification data(2) adjustment of the shooting condition at the time of loginThe respective kinds of processing will be described in detail below.

(1) Automatic Registration of Face Identification Data

To correctly identify a user through comparison with already-registeredface identification data even when the shooting environment such asillumination or the user oneself changes, it is desirable to generateand register face identification data from images shot in as many statesas possible. However, there is a limit to making of such states by theuser oneself in practice. Therefore, in the present embodiment, piecesof face identification data are registered at proper timings determinedby the information processing device 10 besides timings when the useroneself requests storing of face identification data. Thus,authentication processing that is less susceptible to the influence ofaging change and situation change is implemented with small troublesomework of the user.

FIG. 8 schematically shows data stored in the registered userinformation holder 130 and the provisionally-registered data holder 156of the face identification data registration section 150, shown in FIG.3. Pieces of data to be used by the face authentication section 114 forauthentication are stored in the registered user information holder 130.This data is one in which the “user identification information” of aregistered user is associated with the “face identification data” asdescribed above. FIG. 8 shows a state in which the “user identificationinformation” and the “face identification data” are each stored inmemory regions 132 a, 132 b, and 132 c corresponding to users “A,” “B,”and “C.”

In the example of FIG. 8, pieces of face identification data acquired atplural timings are shown by plural rectangles overlapping with eachother. Qualitatively, the authentication accuracy becomes more stablewhen more pieces of data are accumulated. When first a new user requestsuser registration, the input acceptor 102 accepts this request via theinput device 6 and notifies the face identification data registrationsection 150 of that effect. Meanwhile, the camera 7 starts shooting ofthe face of this user and the face identification data registrationsection 150 reads out data of a shot image acquired by the imageacquirer 104 from a memory.

Subsequently, the data generator 152 extracts the region of the facefrom the shot image and derives feature data. In addition, the datagenerator 152 associates the feature data with user identificationinformation such as a new online ID accepted by the input acceptor 102from the user and an account given by the device and stores theassociated data in a memory region newly ensured in the registered userinformation holder 130. Also when a user who has already completed userregistration desires registration of face identification data at anothertiming after the user registration, similarly the data generator 152extracts the region of the face from an image shot at this timing toderive feature data and store it in the registered user informationholder 130 as the face identification data.

However, in this case, the face identification data is additionallystored in the corresponding memory region on the basis of useridentification information identified in advance by login processing orthe like. Such additional registration possibly occurs when the useroneself becomes aware of the necessity for registration due to theelapse of a certain amount of time from the previous registration orchange in the state of the user for example.

In addition to these cases, the registration determiner 154 of the faceidentification data registration section 150 determines whether or notto additionally store face identification data by utilizing the timingwhen a user requests login and the login controller 110 executes theauthentication processing for the login. Specifically, when thefirst-stage face authentication or the second-stage face authenticationsucceeds, i.e. when a newly-shot face image is identified as a faceimage of any registered user, feature data based on this face image isstored in the memory region of the identified user if this face imagesatisfies a predetermined condition.

In this case, the feature data that should be newly stored has beengenerated in the authentication processing by the face authenticationsection 114. Therefore, the data generator 152 may acquire this datafrom the face authentication section 114 and store it in the registereduser information holder 130. Here, as the condition for the storing, thedegree of match with the already-registered face identification dataderived at the time of the authentication is utilized. For example, ifthe degree of match is equal to or higher than a predetermined value,additional registration of the relevant feature data is considered toprovide little contribution to diversification of the faceidentification data and therefore this feature data is not stored.

If the degree of match is equal to or lower than a predetermined value,the possibility that the identified user is a different person is leftand thus the feature data is not stored. Moreover, if comparison is madealso with the face identification data of users other than the useridentified as the result of the authentication and even one user that isdifferent from the identified user and has the degree of match equal toor higher than the predetermined value exists, the possibility that thefeature data is the data of this user is left and thus the feature datadoes not have to be stored. This makes it possible to collect faceidentification data with high reliability in various situationsefficiently and continuously without forcing the user to do troublesomework, which stabilizes the authentication accuracy.

Furthermore, in the present embodiment, even though the first-stage faceauthentication and the second-stage face authentication do not succeed,the feature data of a face image obtained from a shot image at the timeis not discarded but held and whether or not to register it is evaluatedin the long term, to thereby make best use of the data obtained at thetime of the login processing. For this purpose, the registrationdeterminer 154 temporarily stores, in the provisionally-registered dataholder 156, the feature data of a face image whose degree of match doesnot satisfy the condition for registration as face identification data.Hereinafter, temporarily saving feature data in this manner will bereferred to as “provisional registration.”

Thereafter, every time login processing is executed and a face imagethat has proven to be of high certainty as an image of a certainalready-registered user through success in authentication or the like isobtained, the feature data thereof is checked also with theprovisionally-registered feature data. The method of this checkprocessing may be the same as the face authentication processingexecuted by the face authentication section 114. The registrationdeterminer 154 stores, in the registered user information holder 130,feature data that satisfies a predetermined condition through pluraltimes of check processing as face identification data to be used forauthentication, to thereby carry out “formal registration” of thisfeature data. For example, if feature data that corresponds with thefeature data of a face image of a specific registered user with highprobability is detected, this feature data is formally registered as theface identification data of this user (e.g. arrow a).

Furthermore, feature data with which an event that this feature datadoes not correspond with the feature data of a face image of anyregistered user occurs with high probability is determined to be of highpossibility to be the data of a user other than the registered users,and is stored in a memory region newly ensured in the registered userinformation holder 130 (e.g. arrow b). That is, the user correspondingto this feature data is considered to be not the registered user, aboutwhich user identification information has been registered, but be ofhigh possibility to request login in the future. Therefore, this user isrecognized as a guest user and the feature data thereof is formallyregistered as face identification data. In FIG. 8, such guest users areshown as “guest 1,” “guest 2,” and “guest 3.”

Once a user is recognized as a guest user, subsequent processing isexecuted equivalently to the processing about the registered users.Specifically, at the time of login processing, the face authenticationsection 114 of the login controller 110 carries out authentication withuse of the face identification data also about the guest user as well asthe registered users. If a login object is authenticated as a guestuser, the online ID displayed in the second-stage authentication likethose shown in FIGS. 6 and 7 is represented as “guest 1,” “guest 2,” . .. or the like. Furthermore, the registration determiner 154 also checksthe feature data of a face image about which authentication as a guestuser has succeeded with the provisionally-registered feature data. As aresult, if correspondence with one of the guest users occurs with highprobability, this feature data is additionally formally registered asthe face identification data of this guest user.

If the face identification data of a guest user has been stored in theregistered user information holder 130, when a new user requests userregistration, whether this user has been registered as the guest user ischecked. Specifically, when the input acceptor 102 accepts the userregistration request and the data generator 152 derives the feature dataof this user from a shot image, the face authentication section 114carries out a face authentication with the face identification data ofthe already-registered guest user and checks whether the faceidentification data whose degree of match surpasses the authenticationthreshold exists.

If the relevant data exists, the face authentication section 114determines the registered face identification data as the data of thisuser and takes measures of associating this face identification datawith newly-registered user identification information and so forth todelete the registration as the guest user. This prevents the occurrenceof inconveniences that part of the data of a newly-registered user isleft as data of a guest user and thus authentication is not correctlycarried out and that face identification data automatically registeredis not associated with the true user identification information.

FIG. 9 is a flowchart showing the procedure of processing of registeringface identification data by the face identification data registrationsection 150. This processing is started every time at least one of thefirst-stage face authentication and the second-stage face authenticationis carried out in the login controller 110. At this time, the faceauthentication section 114 of the login controller 110 executesauthentication processing of checking the feature data of a face regionin a shot image with the already-registered face identification datastored in the registered user information holder 130 and identifying thecorresponding user or guest user on the basis of the degree of matchtherebetween (S40). In the login controller 110, login processing isexecuted as appropriate on the basis of the result of the authenticationprocessing.

Meanwhile, the registration determiner 154 of the face identificationdata registration section 150 acquires the feature data obtained fromthe face image and the authentication result from the faceauthentication section 114. If the authentication result satisfies theregistration condition (Y of S42), the registration determiner 154registers the feature data in the registered user information holder 130(S44). Here, the authentication result includes not onlysuccess/unsuccess of the authentication but also the degrees of matchwith the face identification data of the respective users and therespective guest users. For example if the degree of match with the faceidentification data of a specific registered user falls within apredetermined range as described above, the feature data is registeredin association with the user identification information of this user.

If the feature data of the shot image does not satisfy the registrationcondition in the determination of S42 (N of S42), subsequently whetheror not the feature data satisfies a provisional registration conditionis determined (S46). If the feature data satisfies the provisionalregistration condition (Y of S46), the registration determiner 154provisionally registers the acquired feature data by storing it in theprovisionally-registered data holder 156 (S48). Basically, theprovisional registration condition is that the degree of match having acertain level or higher of possibility that the feature data correspondsto any user is obtained although the feature data does not have so highcertainty as to be allowed to be directly registered as the data of aspecific user.

If even the provisional registration condition is not satisfied, thefeature data is stored in none of memory regions (N of S46). However,the feature data that does not satisfy the registration condition in S42may be provisionally registered in the provisionally-registered dataholder 156 without exception. In this case, the determination of S46 isomitted. In either case, consecutively the registration determiner 154evaluates feature data stored in the provisionally-registered dataholder 156 at the timing. Then, if the feature data that can be storedin the registered user information holder 130 exists, the registrationdeterminer 154 formally registers it by storing it as appropriate (S50).

FIG. 10 is a diagram for explaining a setting example of theregistration condition and the provisional registration condition.Suppose that pieces of the face identification data of the users A, B,and C are registered in the registered user information holder 130. Asdescribed above, also when pieces of the face identification data ofguest users are further registered, the guest users are treatedsimilarly to the registered users. The face authentication section 114checks the feature data of a newly-shot face image with the faceidentification data of each user and thereby calculates the degrees ofmatch with the respective users.

Rectangles under the respective users “A,” “B,” and “C” in FIG. 10represent the range of the degree of match in the vertical direction.The bottom side indicates 0 points as an example of the lowest degree ofmatch and the top side indicates 100 points as an example of the highestdegree of match. If plural pieces of face identification data existabout each user, the degrees of match may be individually calculated orthe degree of match may be calculated on each user basis by statisticalprocessing.

Qualitatively, if the degree of match with a specific user is high, thefeature data is directly registered as the face identification data ofthis user. This corresponds to the “registration condition” in S42 ofFIG. 9. However, as described above, newly storing feature data that istoo close to the already-registered face identification data isconsidered to hardly contribute to improvement in the authenticationaccuracy. For this reason, in the example shown in FIG. 10, a lowerlimit Th1 and an upper limit Th2 are set as the registration conditiongiven to the degree of match. For example, if the degree of match withthe face identification data of the user A is from Th1 to Th2 inclusive,the feature data is registered as the face identification data of theuser A.

However, with the exception of the case in which the feature datasatisfies this condition regarding plural users, the feature data is notdirectly registered but provisionally registered once and thereby whichuser corresponds to this feature data is evaluated over time.Furthermore, the lower limit Th1 of the registration condition may bethe same as the authentication threshold for determining authenticationsuccess in face authentication carried out by the face authenticationsection 114 or may be different. For example, by employing the degree ofmatch higher than the authentication threshold as the lower limit Th1 ofthe registration condition as shown in FIG. 10, whether or not toregister the feature data may be determined on the basis of the morestrict criterion than that of the login processing.

Furthermore, if the degree of match that does not reach the lower limitTh1 of the registration condition but is close to it is obtained, thatthe degree of match is equal to or higher than Th0 and is lower than Th1is regarded as the “provisional registration condition” in S46 of FIG. 9as shown in FIG. 10 for example and the feature data is provisionallyregistered. At this time, the feature data may be stored in theprovisionally-registered data holder 156 in the state in which thefeature data is associated with none of users as shown in FIG. 8 or maybe associated with an object user who satisfies the provisionalregistration condition. In the latter case, the degree of match may bechecked only with the feature data of a face image of this associateduser in subsequent evaluation. Furthermore, because long-term evaluationis made on the provisionally-registered feature data, feature data maybe provisionally registered without exception even when plural objectusers who satisfy the provisional registration condition exist.

The registration condition and the provisional registration conditionshown in FIG. 10 are one example and other various settings will also beavailable. For example, feature data having the degree of match lowerthan the lower limit Th1 of the registration condition as describedabove may all be provisionally registered. Alternatively, feature datawhose degrees of match with all users are equal to or lower than apredetermined value may be further regarded as a provisionalregistration target. This is because such data can become a candidatefor the face identification data of the above-described guest user.

FIG. 11 is a flowchart showing the procedure of processing of formallyregistering provisionally-registered data by the registration determiner154 in S50 of FIG. 9. First, the registration determiner 154 comparesthe feature data of a face image having a high possibility of being aface image of a specific user, such as a face image whose degree ofmatch with already-registered face identification data is equal to orhigher than a predetermined value, among face images shot and extractedin S40 of FIG. 9 with feature data stored in theprovisionally-registered data holder 156, and calculates the degree ofmatch (S60). The former face image may be identified under the samecondition as the condition for success in authentication. Furthermore,this comparison processing may be similar to the processing ofcalculating the degrees of match between the feature data of a faceimage in a shot image and the face identification data of the respectiveusers by the face authentication section 114 at the time of loginprocessing.

Then, the registration determiner 154 determines whether or not areliability update condition that the degree of match is equal to orhigher than a predetermined value or is equal to or lower than apredetermined value is satisfied (S62). Here, the reliability isfrequency representing the magnitude of the probability that the targetfeature data is the data of an already-registered user or a guest useror the magnitude of the probability that the target feature data is thedata of none of the users, i.e. is the data of an unregistered user. Forexample, the reliability is represented by the number of times ofsatisfying of the reliability update condition. If feature datasatisfies the reliability update condition (Y of S62), the registrationdeterminer 154 updates the reliability of the relevant user or anunregistered user (S64).

After updating the reliability, the registration determiner 154 checkswhether or not a formal registration condition is satisfied,specifically for example this reliability reaches a predetermined value(S66). If the formal registration condition is satisfied (Y of S66), theregistration determiner 154 formally registers the feature data bystoring it in the relevant memory region of the registered userinformation holder 130 (S68). If the reliability update condition is notsatisfied in S62 (N of S62), the reliability does not change andtherefore the determination of the formal registration condition isomitted. Such reliability update determination and determination aboutwhether or not to allow formal registration are made about each piece offeature data (N of S70 and S60 to S68). When the evaluation is completedabout all pieces of provisionally-registered feature data (Y of S70),the processing is ended.

FIG. 12 is a diagram for explaining a setting example of the reliabilityupdate condition used in S62 of FIG. 11. Suppose that, similarly to FIG.10, pieces of the face identification data of the users “A,” “B,” and“C” are registered in the registered user information holder 130 and therange of the degree of match with the feature data of a face imagedetermined to be of high certainty as a face image of the user isrepresented by each rectangle. Also when guest users have beenregistered, the guest users are treated similarly to the registeredusers. As shown in FIG. 12, if the degree of match with the face imageis equal to or higher than a threshold th1, it is determined that thereliability update condition is satisfied and the reliabilitycorresponding to the user is updated. For example, if the degree ofmatch with a face image determined as the user A is equal to or higherthan the threshold th1, the reliability corresponding to the user A isupdated. Furthermore, also when the degree of match with the face imageis equal to or lower than a threshold th0, it is determined that thereliability update condition is satisfied. In this case, the reliabilitycorresponding to the unregistered user is updated.

FIG. 13 schematically shows a data example of the reliability. Theprovisionally-registered feature data each holds the reliabilitycorresponding to the respective users (users “A,” “B,” and “C”) and the“unregistered user” as shown in FIG. 13. Furthermore, every time thereliability update condition like that shown in FIG. 12 is satisfied,the reliability of the corresponding user or the unregistered user isupdated. At the timing when the reliability of any user reaches apredetermined value P, the feature data is formally registered as theface identification data of the corresponding user or a new guest user.In the case of FIG. 13, the reliability is shown as a parameter thatbecomes higher when the possibility that the feature data is the data ofthe corresponding user is higher. However, this does not intend to limitthe characteristics of the reliability thereto.

Due to the above configuration, the feature data of a face image whoseregistration is avoided because of low certainty at the time of shootingcan also be given opportunities to be used by being formally registereddepending on the subsequent evaluation result. In particular, the piecesof the face identification data stored in the registered userinformation holder 130 are updated at any time by the above-describedautomatic registration processing and so forth. Therefore, there is apossibility that the reliability rises by continuously evaluating thefeature data by utilizing the authentication processing and the featuredata is registered. Furthermore, by evaluating provisionally-registereddata on the basis of a shot face image, the reliability of the datafalling within the allowable range of the degree of match can beproperly updated even when the already-registered face identificationdata do not change.

As a result, a situation in which data is not registered due totemporary change in the shooting environment or slight differenceattributed to noise or the like can be avoided. In addition, it becomespossible to continuously register high-accuracy data obtained by addingchanges over a comparatively-long period, such as the growth andappearance change of the user oneself. Furthermore, by permitting aguest user, provided that feature data does not correspond with anyregistered user in the long-term evaluation, and registering the featuredata, the possibility that an unregistered user is erroneouslyrecognized as any of the registered users in subsequent login processingcan be made low.

Feature data in which the frequency of satisfying of the reliabilityupdate condition is low and the reliability of any user does not reachthe predetermined value P even after the elapse of a predeterminedperiod may be deleted from the provisionally-registered data because thepossibility that the feature data is unsuitable for registration due tothe existence of a defect in the data itself is high. For example, anupper limit may be set in the number of pieces of feature data that canbe provisionally registered. Furthermore, when the number of pieces ofprovisionally-registered feature data reaches the upper limit, featuredata that has gone through a long period from its provisionalregistration and has a large difference between the reliability and thepredetermined value P may be discarded and new feature data may bestored. Thus, the memory capacity of the provisionally-registered dataholder 156 can be made limited.

(2) Adjustment of the Shooting Condition at the Time of Login

In the case of comparing registered face identification data with thefeature data of a face image obtained at the time of login in the faceauthentication processing described thus far, if the shootingenvironments of the respective images are different as described above,it may be impossible to obtain the degree of match surpassing theauthentication threshold even though these images are images of the sameuser attributed to difference in the image characteristics such as theluminance distribution. Furthermore, if the authentication threshold islowered in consideration of such a situation, correspondence with adifferent user is readily caused.

In many cases, a general camera has a function to automatically adjustthe shooting condition (including the correction condition) such as theexposure, the diaphragm, the gamma value, and the white balance so thatthe optimum shot image may be obtained according to the illuminationenvironment of the moment and so forth. For example, a technique isprevailing in which the appearance of an image is improved and theaccuracy of detection of a target object is improved by functions ofauto exposure, auto gain control, and so forth. However, such functionsare intended to adjust the overall balance on each image basis andtherefore do not effectively work for the above-described faceauthentication, which is premised on comparison of a partial regionbetween two images. On the contrary, possibly these functions increasethe difference in the image characteristics and cause adverse effects.

Therefore, the shooting condition controller 106 of the informationprocessing device 10 controls the shooting condition of the camera 7 sothat the image characteristics of a face image shot at the time of loginprocessing may get close to the image characteristics of a face image asthe source of registered face identification data. At this time, if theface image itself is employed as the face identification data, the imagecharacteristics of this image may be employed as the target. If dataother than the shot image, such as the shapes and sizes of parts of theface and difference from an average face, is employed as the faceidentification data, data relating to the image characteristics of theoriginal face image is saved in the registered user information holder130 in addition to the face identification data and is read out at thetime of the adjustment.

FIG. 14 is a diagram for explaining the basic adjustment operation ofthe shooting condition controller 106. First, the face identificationdata of a registered user is stored in the registered user informationholder 130. The face identification data is stored in association withuser identification information as shown in FIG. 8 in practice.Furthermore, the face identification data is presentationally shown as aface image 182 in FIG. 8 although being not necessarily the face imageitself as described above. On the other hand, if a face region extractedfrom an image shot in login processing is e.g. a face image 180 a, theluminance is totally different compared with the registered face image182 and therefore there is a possibility that the authentication failseven though these face images are face images of the same user.

Therefore, the shooting condition controller 106 acquires the data ofthe face image 180 a from the face authentication section 114 anddecides the adjustment target of the shooting condition so that theimage characteristics of the face image 180 a may get close to the imagecharacteristics of the face image 182 as the source of the faceidentification data. As the image characteristics, any parameter used inimage analysis or an image processing technique may be employed. Forexample, any of the luminance distribution, the frequency distribution,the color distribution, the grayscale distribution, and so forth or acombination of them may be employed. Besides the image characteristics,characteristics of the figure in the image, such as the distribution ofparts of the face, may be utilized. In the example of FIG. 14, theshutter speed and the diaphragm are adjusted and the gain is lowered insuch a direction as to lower the exposure as shown by an arrow.

Then, the camera 7 is made to perform shooting with gradual change inthe shooting condition by every predetermined amount so that theshooting condition may reach the decided target. At this time, everytime a shot image is obtained, the degree of match between the faceimage thereof (e.g. face image 180 b) and the registered face image 182(or face identification data thereof) is checked, and the face imagedeemed to be shot under the optimum condition (e.g. face image 180 c) isacquired. The path along which the shooting condition reaches theadjustment target is not limited to a one-direction path like that shownin FIG. 14 and the shooting condition may be brought closer to thetarget through swinging toward the higher and lower sides.

Alternatively, the plural face images 180 b, 180 c, and 180 d in apredetermined condition range may be acquired at one time by bracketshooting and thereafter the optimum face image 180 c may be selected.Furthermore, depending on the image characteristic on which attention isfocused, the optimum shooting condition may be obtained by calculationwithout gradually changing the shooting condition. In this case, theamount of adjustment of the shooting condition is calculated inaccordance with a calculation expression prepared in advance on thebasis of the difference between the image characteristic of the faceimage 182 as the source of the face identification data and the imagecharacteristic of the shot face image 180 a, and a face image shot underthe condition after the adjustment is employed. It will be understood bythose skilled in the art that this calculation expression is variouslygiven depending on the image characteristic.

FIG. 15 is a flowchart showing the procedure of processing of carryingout face authentication by the face authentication section 114 whileadjusting the shooting condition of the camera 7 by the shootingcondition controller 106. Typically this processing is executed in thefirst-stage face authentication, which is the initial stage of shooting.However, this processing can be applied also to the second-stage faceauthentication. First, the camera 7 performs shooting under a conditionin accordance with the initial setting and the face authenticationsection 114 acquires the shot image (S80 and S82). The shootingcondition of this time may be decided by using a general automaticadjustment function possessed by the camera 7.

Next, the face authentication section 114 compares the feature data ofan image of a face region extracted from the shot image with the faceidentification data of each registered user stored in the registereduser information holder 130 and calculates the degree of match asdescribed above (S84). Then, if the registered user about which thedegree of match surpasses the authentication threshold exists (Y ofS86), the face authentication section 114 determines that theauthentication has succeeded without a problem and makes transition tothe subsequent processing such as the second-stage face authenticationand login processing (S88).

On the other hand, if the user whose degree of match surpasses theauthentication threshold does not exist even though the face is detected(N of S86), the shooting condition controller 106 starts the processingof adjusting the shooting condition (Y of S90 and S94 to S98).Specifically, first if plural face images are extracted from the shotimage, the shooting condition controller 106 decides a face imageemployed as the comparison object among them on the basis of apredetermined rule (S94). For example, the shooting condition controller106 selects the face image from which the highest value is obtained asthe degree of match calculated in S84. Alternatively, the shootingcondition controller 106 may select the face image having the largestsize of the face region, the face image estimated to be obtained byshooting the face closest to the camera 7, the face image at theposition closest to the center of the shot image, or the like.

Next, the shooting condition controller 106 decides a face imageemployed as the target of image characteristic approximation among theface images of the registered face identification data on the basis of apredetermined rule (S96). For example, the shooting condition controller106 selects any of the face image with the latest registration timing,the face image having a high frequency of surpassing of theauthentication threshold in the past authentication processing, the faceimage having the highest degree of match with the face image of thecomparison object decided in S94, the face image with a face orientationclosest to that of the face image decided in S94, and so forth. Pluralcriteria may be employed as the selection rule. In the registered userinformation holder 130, information according to the employed criterionis stored as appropriate in association with the face identificationdata. Furthermore, the face image with which the degree of match has notsurpassed a predetermined value in the past authentication processingmay be excluded from the choices.

Next, the shooting condition controller 106 compares the face image ofthe comparison object decided in S94 in the shot image with the faceimage that is decided in S96 and is employed as the target on the basisof the image characteristic like the above-described one to therebydecide the adjustment target of the shooting condition and change theshooting condition of the camera 7 in accordance with the result of thedecision (S98). Then, the face authentication section 114 acquires animage shot under the condition after the adjustment to thereby comparethe feature data of an image of a face region with the faceidentification data of each registered user stored in the registereduser information holder 130 again and calculate the degree of match(S84). If a user whose degree of match surpasses the authenticationthreshold is obtained through this process (Y of S86), the faceauthentication section 114 determines that the authentication hassucceeded (S88).

If a user whose degree of match surpasses the authentication thresholddoes not exist (N of S86) and there is still room for adjustment in theshooting condition (Y of S90), the shooting condition controller 106adjusts the shooting condition again (S94 to S98). It is also possibleto omit the processing of S94 and S96 depending on the degree of matchcalculated in S84. Specifically, if the degree of match changes in theimprovement direction, it is estimated that the policy of the adjustmentdecided first is correct and therefore only changing the shootingcondition may be carried out with omission of the processing of S94 andS96.

If the degree of match does not change or is deteriorated, the faceimage of the comparison object and the face image employed as the targetmay be reselected in S94 and S96 or the adjustment target of theshooting condition may be changed in S98. Such adjustment of theshooting condition and authentication are repeated and, if a user whosedegree of match surpasses the authentication threshold is obtained (Y ofS86), the face authentication section 114 determines that theauthentication has succeeded (S88). If the adjustment range of theshooting condition reaches a predetermined limit (N of S90) withoutobtainment of a user whose degree of match surpasses the authenticationthreshold (N of S86), possibly the login object is not the registereduser and therefore the face authentication section 114 determines thatthe authentication has resulted in unsuccess and ends the processing(S92).

In the case of continuously shooting images while changing the shootingcondition to a predetermined adjustment range by bracket shooting,authentication processing is executed in order of acquisition of theshot image in S84 and S86 and success and unsuccess of theauthentication are determined on the basis of the shot image with whichthe highest degree of match is obtained. Furthermore, in theabove-described example, when authentication of even one personsucceeds, the adjustment of the shooting condition is not furthercarried out and the authentication processing is ended. However, while aface image about which authentication has not succeeded is left, theadjustment of the shooting condition may be continued as long as thereis room for adjustment. In this case, any of the left face images isemployed as the face image decided as the comparison object in S94. Inthis case, without selecting the face image based on the above-describedcriterion in S94, all face images may be used for the adjustment e.g. bysequentially employing the face images included in the shot image as thecomparison object from an edge side. At this time, the authenticationtarget may be limited to the face image used for this adjustment orother face images may also be regarded as the target of theauthentication processing.

Furthermore, in the procedure shown in FIG. 15, the efficiency of theadjustment processing is sought by narrowing down the registered imagesemployed as the target in S96 to one image. However, all registeredimages may be sequentially employed as the target and the shootingcondition may be adjusted from various angles. That is, the registeredface image is extracted to be employed as the target one by one in S96and the shooting condition adjustment and the authentication processingin S98 and the subsequent steps are executed. In this case, if theadjustment range of the shooting condition reaches a predetermined limit(N of S90) without obtainment of a user whose degree of match surpassesthe authentication threshold (N of S86), the next registered image isemployed as the target, with the face image of the comparison objectleft unchanged, and the same processing is repeated. Then, theprocessing is ended when the authentication results in unsuccess evenafter the adjustment is carried out about all registered images.

Due to such adjustment processing, even when the image characteristicsof an image shot at the time of login differ from the imagecharacteristics of the registered face identification data because ofchange in the shooting environment such as illumination, it is possibleto suppress the occurrence of an inconvenience that the authenticationaccuracy is deteriorated attributed to the difference and the user hasdifficulty in login. Furthermore, because the adjustment is carried outaccording to already-registered data, troublesome work of retryingshooting until data allowing proper authentication is obtained at thetime of data registration is alleviated.

Therefore, for example if the adjustment processing is combined with theabove-described automatic registration of face identification data, evenwhen a registration object is decided comparatively easily, thedeterioration of the authentication accuracy due to this can beprevented. Furthermore, in this case, an image shot through conditionadjustment becomes the registration object and therefore the imagecharacteristics of the registered data become uniform by themselves. Asa result, obtainment of a high value as the degree of match with data ofa correct user is facilitated and hence the authentication accuracy canbe improved.

The present disclosure is described above based on the embodiment. Itwill be understood by those skilled in the art that this embodiment isexemplification and various modification examples are possible incombinations of the respective constituent elements and the respectiveprocessing processes thereof and such modification examples are alsowithin the scope of the present disclosure.

For example, the target of use of the present embodiment is not limitedto user login. Specifically, the present embodiment can be applied toany technique irrespective of the field and use purpose thereof as longas the technique is to carry out some kind of determination,discrimination, or detection by checking the degree of match between aregistered image and a newly-acquired image. For example, theabove-described data automatic registration technique and shootingcondition adjustment technique can be applied even to a case of onlyidentifying who or what a subject is, and effects similar to thosedescribed in the present embodiment can be obtained.

The present disclosure contains subject matter related to that disclosedin Japanese Priority Patent Application JP 2014-211057 filed in theJapan Patent Office on Oct. 15, 2014, the entire content of which ishereby incorporated by reference.

What is claimed is:
 1. An information processing device comprising: animage acquirer configured to acquire a shot image of a user from animaging device; a registered user information holder configured to holdface identification data of a registered user; a face authenticationsection configured to execute authentication processing of a face imagein the shot image by using the face identification data held in theregistered user information holder; a shooting condition controllerconfigured to adjust a shooting condition to bring a characteristic ofthe face image close to a characteristic of a face image of anacquisition source of the face identification data and make the imagingdevice perform shooting in order to improve an authentication result inthe face authentication section; and an information processing sectionconfigured to execute information processing on a basis of anauthentication result of a face image obtained by adjustment.
 2. Theinformation processing device according to claim 1, wherein theregistered user information holder further holds characteristic data ofthe face image of the acquisition source of the face identification datain association with the face identification data, and the shootingcondition controller decides an adjustment target of the shootingcondition by comparing characteristic data acquired from the face imagein the shot image with characteristic data read out from the registereduser information holder.
 3. The information processing device accordingto claim 1, further comprising: a face identification data registrationsection configured to update the face identification data on a basis ofthe face image obtained by adjustment when degree of match between theface image obtained by adjustment and the face identification datasatisfies a predetermined condition.
 4. The information processingdevice according to claim 1, wherein the shooting condition controllerselects face identification data employed as a target of characteristicapproximation from a plurality of pieces of face identification dataheld by the registered user information holder on a basis of any ofregistration timing, frequency of success in authentication in pastauthentication processing, the degree of match with the face image inthe shot image, and degree of similarity of a face orientation to theface image.
 5. The information processing device according to claim 1,wherein the shooting condition controller sequentially sets a pluralityof pieces of face identification data held by the registered userinformation holder as a target of characteristic approximation andadjusts the shooting condition to make the imaging device performshooting.
 6. The information processing device according to claim 1,wherein the shooting condition controller selects a face image employedas an object of characteristic approximation from a plurality of faceimages in the shot image on a basis of any of the degree of match withthe face identification data, area of a face region, distance of a facefrom the imaging device, and a position in the shot image.
 7. Theinformation processing device according to claim 1, wherein the shootingcondition controller sequentially sets a plurality of face images in theshot image as an object of characteristic approximation and adjusts theshooting condition to make the imaging device perform shooting.
 8. Theinformation processing device according to claim 1, wherein the shootingcondition controller repeats adjustment of the shooting condition untilthe degree of match between the face image and the face identificationdata surpasses a predetermined authentication threshold andauthentication succeeds.
 9. The information processing device accordingto claim 1, wherein the shooting condition controller makes the imagingdevice continuously perform shooting while changing the shootingcondition in a stepwise manner, and determines success and unsuccess inauthentication by using a face image whose degree of match with the faceidentification data is highest among face images in a plurality ofobtained shot images.
 10. The information processing device according toclaim 1, wherein the shooting condition controller decides amount ofadjustment of the shooting condition on a basis of difference betweenthe characteristic of the face image of the acquisition source of theface identification data and the characteristic of the face image in theshot image.
 11. An information processing method comprising, by aninformation processing device: acquiring a shot image of a user from animaging device; reading out face identification data of a registereduser stored in a storage device and carrying out authentication of aface image in the shot image by using the face identification data;adjusting a shooting condition to bring a characteristic of the faceimage close to a characteristic of a face image of an acquisition sourceof the face identification data and making the imaging device performshooting in order to improve an authentication result in the carryingout authentication; and executing information processing on a basis ofan authentication result of a face image obtained by adjustment.
 12. Acomputer program for a computer, comprising: acquiring a shot image of auser from an imaging device; reading out face identification data of aregistered user stored in a storage device and carrying outauthentication of a face image in the shot image by using the faceidentification data; adjusting a shooting condition to bring acharacteristic of the face image close to a characteristic of a faceimage of an acquisition source of the face identification data andmaking the imaging device perform shooting in order to improve anauthentication result in the carrying out authentication; and executinginformation processing on a basis of an authentication result of a faceimage obtained by adjustment.